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Abstract 


CALIPSO aerosol backscatter enhancement in the transition zone between clouds and 
clear sky areas is revisited with particular attention to effects of data selection based on 
the confidence level of cloud-aerosol discrimination (CAD). The results show that 
backscatter behavior in the transition zone strongly depends on the CAD confidence 
level. Higher confidence level data has a flatter backscatter far away from clouds and a 
much sharper increase near clouds (within 4 km), thus a smaller transition zone. For high 
confidence level data it is shown that the overall backscatter enhancement is more 
pronounced for small clear-air segments and horizontally larger clouds. The results 
suggest that data selection based on CAD reduces the possible effects of cloud 
contamination when studying aerosol properties in the vicinity of clouds. 

Keywords: backscatter enhancement, transition zone, twilight zone, CALIPSO, aerosol, 
remote sensing 
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1. Introduction 


It is widely recognized that the characteristics of clear sky areas tend to change near 
clouds, creating transition zones between cloudy and clear areas. The transition zones are 
well known for their systematic enhancement of brightness or aerosols optical depth 
retrieved from satellites (Ignatov et ah, 2005; Loeb and Manalo-Smith, 2005; Matheson 
et ah, 2005; Zhang et ah, 2005; Koren et ah, 2007; Loeb and Schuster, 2008), airborne 
(Redemann et ah, 2009; Su et ah, 2008), or ground instruments (Koren et ah, 2007). 

Such zones can extend from hundreds of meters up to tens of kilometers (Perry and 
Hobbs, 1996; Lu et al., 2003; Charlson et ah, 2007; Koren et al., 2007). 

Such systematic enhancements in the transition zones seem to imply changes of aerosol 
properties near clouds (Bar-Or et al., 2011, Koren et al., 2008a; Su et al., 2008; Clark et 
al., 2002; Hoppel et al., 1986). Examination of aerosol property variations in such zones 
would help improve our understanding of aerosol-cloud interactions and allow more 
accurate estimations of the direct and indirect effects of aerosols in radiative budgets and 
climate forcing (Koren et al., 2008b; Medeiros et al., 2008). However, artifacts inherent 
in satellite-based signal detection and processing can also contribute to the apparent 
enhancements. These artifacts include the blurring effects of imaging systems (Qiu et al., 
2000), three-dimensional (3D) radiative effects (e.g., Wen et al., 2007; Marshak et al., 
2008; Kassianov and Ovchinnikov, 2008) from adjacent clouds, and cloud contamination 
(Ignatov et al., 2005; Zhang et al., 2005). Clearly, eliminating or reducing the effects of 
these artifacts is needed for correct retrievals of near-cloud aerosol parameters. 

The CALIOP (Cloud-Aerosol Lidar with Orthogonal Polarization) lidar onboard the 



CALIPSO (The Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation) 
satellite (Winker et ah, 2007) is an active instrument and is impervious to the blurring 
effects and the 3D effects of adjacent clouds that influence passive remote sensing 
instruments. As a result, it is especially well-suited for studying aerosol-cloud 
interactions and aerosol properties in the transition zone. Using CALIPSO data, Tackett 
and Di Girolamo (2009) and Vamai and Marshak (2011a) have observed near-cloud 
backscatter enhancements in the transition zone free of the artifacts of passive 
instruments. 

Yet, the acquired aerosol data can still be cloud contaminated even in the lidar data. This 
is because clouds normally have no clear boundaries. Large-eddy simulations (Charlson 
et ah, 2007) have shown that activated droplets can inhabit in much larger areas than the 
areas normally called cloudy. As described in studies of large transition zones using 
AERONET and MODIS data (Koren et ah, 2007) and studies of cloud fields (Charlson et 
ah, 2007; Koren et ah, 2009), undetected wispy clouds, together with hydrated aerosols, 
are major contributors to the enhancement phenomena. Like all other remote sensing 
instruments, the CALIOP’s binary identification of cloudy and cloud- free pixels is based 
on thresholds that depend on instrument characteristics and algorithms. Naturally, this 
binary definition cannot fully represent the continuum from obviously cloudy to 
obviously clear areas. The attempt to unambiguously distinguish between clear and 
cloudy areas in remote sensing data therefore results in declaring some pixels clear even 
if they contain some clouds. Hence, cloud contamination is an unavoidable feature in any 
study of aerosol property variations using remotely sensed aerosol data near clouds. 
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To reduce the impact of cloud contamination, median values are used to characterize 
aerosol backscatter behaviors (Tackett and Di Girolamo, 2009; Varnai and Marshak, 
2011a), because medians are less influenced than means by any extreme high backscatter 
values arising from cloud contamination. Still, the extent of undetected clouds affecting 
or masking features from “true” aerosols is unclear and should be a subject of further 
studies. 

This paper examines CALIOP observations of near-cloud aerosol properties, especially in 
relationship to the CAD score in the aerosol layer product. As discussed in detail in 
Section 2, the CAD score provides a measure of the confidence for each identification 
made (Liu et al., 2009). 

This paper also suggests that CAD can help reduce the possible impact of cloud 
contamination if only aerosol data with CAD values beyond a certain threshold is used. 
This threshold has to be carefully chosen based on the balance between the needs for 
capturing purely aerosol physics with less cloud contamination and for using all- 
encompassing aerosol datasets. 

The paper is organized as follows: First, Section 2 discusses the physical meaning of 
CAD scores, then Section 3 describes the methodology used in this study. Next, Section 
4 discusses CAD-dependent aerosol behaviors observed near clouds. Finally, Section 5 


summarizes the results. 



2. A brief overview of CAD scores 


The operational CALIOP data processing algorithm determines the Cloud-Aerosol 
Discrimination (CAD) score using the equation 

CAD(X) ~ PclOUd k P *=r°sol (X } x j ()()% (1) 

Pclo ad ( X > k P aerosol (X) 

where X is the array of input measurements (i.e., altitude, 532 nm lidar backscatter, and 
the ratio of backscatters at 1064 and 532 nm at 5 km horizontal resolution), Aloud and 
Aerosol are probability density functions (PDFs) for clouds and aerosol, respectively, and k 
is the ratio of occurrence frequencies of cloud and aerosol layers. Because CAD scores 
are positive when Aloud > k Aerosol and negative when Aloud < k Aerosol, layers with a 
positive CAD score are classified as cloud, and layers with a negative CAD value are 
classified as aerosol. While the sign of CAD scores determines the type of a feature, the 
magnitude, which is essentially a probability difference, describes how much more likely 
the feature is to be of one type than of the other. 

The probability-difference nature of CAD suggests us a view angle to look at the cloud 
contamination issues in the observations of transition zone using CALIOP lidar. For 
aerosols such as those in traditionally defined clear areas far away from the transition 
zone, their absolute CAD values are relatively large because of their distinct differences 
in their cloud/aerosol probabilities. For the undetected cloud fragments and large 
hydrated aerosols such as those in cloud-free areas of transition zones, their absolute 
CAD values should be relatively small because of the small difference in their 
cloud/aerosol probabilities. This tendency may be understood by considering a virtual 



experiment of adding cloud droplets into an aerosol volume. The addition of cloud 
droplets into an aerosol volume would increase the weight of cloud property for that 
volume, causing /? c ioud to increase in Eq. (1); aerosols contaminated by undetected clouds 
therefore tend to reduce probability difference or CAD value. This implies that using only 
aerosol data with high confidence values (large negative CAD scores) reduces cloud 
contamination by excluding some observations affected by cloud particles. From the 
other hand, using only high confidence data may exclude some large hydrated aerosol 
particles from the analyses. 

3, Methodology 

The main objective of this research is to improve our understanding of near-cloud aerosol 
behaviors by examining their dependence on associated CAD values. Specifically, the 
study examines the relationships between CAD scores and near-cloud aerosol parameters 
such as integrated attenuated total backscatter (y'>_) and integrated attenuated total color- 
ratio (%'); in addition, it examines the way these relationships depend on the size of clear- 
air segments and the horizontal size of clouds. 

The study has included the analysis of monthlong CALIOP Version 2 nighttime datasets 
from each of the four seasons. (The comparison between Versions 2 and 3 data is 
discussed in the Appendix). Due to the similarity of behaviors in all months, however, 
the paper only presents results for a single month ranging from March 9 to April 10, 

2008. The study considers all data over oceans free of sea ice, but excludes data over 
land for simplicity in data interpretation and for some limited environmental similarity. 
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The methodology is similar to that of Vamai and Marshak (2011a). For this study, 
CALIOP Level 1 attenuated backscatter (yV) profiles are integrated from 90 m to 3 km 
above mean sea level at 333 m resolution, and values of integrated attenuated total color- 
ratio (x') are computed as yW/y 532 . We focus on low altitudes because low warm clouds 
have the strongest impact on aerosols though the results are close to those integrated to 
15 km (Varnai and Marshak, 2011a). The analysis uses clear profiles that are identified 
as cloud free in both the 0.333 km and 1.0 km operational cloud layer products, and the 
CAD score reported in the 5 km aerosol layer product is used to sort data. A 333 m 
backscatter profile is selected and used if the following conditions are all met: (a) The 
height of nearest cloud top is below 3 km. (b) The height of the top aerosol layer in the 
profile is below 3 km. (c) The distance between the two neighboring cloud edges is 
greater than 5 km, except in Section 4.3.1 where the distance between two neighboring 
cloud is a variable. 

Median values provide more accurate characterizations of data with large outliers and are 
used in the analysis of this paper. The uncertainties of median values are estimated using 
the bootstrapping algorithm (Efron and Gong, 1983). 

4. Observations and results 

To illustrate and understand the influence of CAD-based data selection, let us start by 
examining the correlations between CAD values and backscatter strengths for two 
distance ranges from cloud edges, 0.33-7.0 km and 7.0-20 km (Figure 1). The results 





show three notable tendencies. First, backscatter is higher closer to clouds. Earlier 
studies (e.g., Tackett and Di Girolamo 2009; Koren et al. 2007) attributed such higher 
backscatter values near clouds to processes such as aerosols swelling in the humid air 
around clouds, and to the increased presence of undetected cloud particles near clouds. 
Second, the attenuated backscatter y ' 532 decreases with increasing confidence levels, 
especially for CAD scores beyond -60. The decrease can be explained by the fact that 
aerosol layers are identified with a higher confidence if backscatter is low, because 
clouds tend to be optically thicker than aerosols. Third, the backscatter enhancement 
near clouds decreases for increasing confidence levels (Figure lb). This occurs because 
much of the enhancement comes from swollen aerosols and undetected cloud drops, and 
neither of these is too frequent in high-confidence aerosol data, given that large particle 
size of swollen aerosols and cloud drops reduces aerosol detection confidence. 

4.1. Number of lidar profiles with high confidence aerosol detection 
Figure 2 examines the number of cloud-free 333 m-resolution lidar profiles that contain 
aerosol layer(s) detected in six CAD ranges. The six CAD ranges are selected arbitrarily, 
to characterize y' S 32 behaviors that change with CAD for CAD values beyond -60 (Figure 
la.). Figure 2a reveals that profile numbers increase near clouds for all six CAD ranges. 
This occurs because both short and long cloud-free segments yield profiles that are close 
to clouds, but only the (fewer) long cloud-free segments can yield profiles that are far 
away from clouds. 

In contrast, Figure 2b shows that the fraction of profiles with high confidence aerosol 
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detections decreases near clouds dramatically. For example, aerosol detection confidence 
reaches the highest maximum level (-99, -100) for 50% of clear profiles far away from 
clouds, but only for 30% close to clouds. Naturally, this implies that relatively more low 
confidence aerosols are detected near clouds than far away from clouds. The decreasing 
ratio of high confidence data near clouds indicates that the properties of aerosols become 
more similar to those of clouds when cloud edges are approached. This can be explained 
by the swelling of hydrated aerosols and/or cloud-contamination near clouds, and can 
provide a different perspective for understanding the characteristics of the transition zone. 

4.2. CAD dependent backscatter and color ratio enhancements near clouds 

In this section we further examine the dependence of integrated attenuated backscatter 

y 532 and color ratio %' on CAD score. 

Figure 3a shows y ' 532 as a function of distance to clouds for six CAD ranges. In 
agreement with Figure 1 and with previous studies (Tackett and Di Girolamo, 2009; 
Vamai and Marshak, 2011a), the figure shows that integrated attenuated backscatter is 
enhanced near clouds. The results also agree with Figure 1 in that higher confidence data 
tends to have lower backscatter values. In addition, the figure indicates that in 
comparison to all aerosol data, high confidence data are flatter far away from clouds and 
have larger enhancements close to clouds. For example, between 5 km and 20 km the 
enhancement for the highest confidence data (-100 < CAD < -99) is only about 1/3 of the 
enhancement for all data (-100 < CAD < 0), whereas between 333 m and 5 km it is 1.7 
times the value for all data. 
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Similar features also appear for attenuated total color ratios (Figure 3b). As seen in 
Figure 3b, %' values for all CAD ranges increase as the distance to cloud decreases, 
indicating that aerosol size and/or concentration generally increase near clouds. High 
confidence level data (with large values of |CAD|) tends to have lower color ratios, 
because of the exclusion of highly humidified aerosols and cloud droplets that are large in 
size and result in low |CAD| scores. Moreover, the rate of color ratio increase far away 
from clouds (x > 5 km) is only about half as big for high confidence data as that for all 
data without CAD selection, whereas near clouds (x < 5 km) this rate is about twice as 
big for high confidence data as that for all data without CAD selection. 

Such new features regarding CAD-dependent variations in backscatter y ' 532 and attenuated 
color-ratio %' are especially important because they may lead to different interpretations 
of the observed aerosol data and to different retrieved aerosol parameters. We note that, 
on one hand, excluding low-confidence aerosol data makes a dataset less comprehensive 
as some data from cloud-like larger aerosol particles may also be excluded; on the other 
hand, results derived from the high-confidence data are more clearly representative of 
only aerosols, without significant interference from undetected cloud particles. 

A remarkable new finding seen in Figure 3 a is the sharp increase in the near-cloud 
enhancement in y 5 32 and %' for the highest confidence level aerosol data. Considering that 
aerosol physical/optical properties are the most distinct from those of clouds for the 
highest confidence level aerosol data (CAD scores between -99 and -100), the sharp 
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enhancement observed for the highest confidence data is likely the most representative of 
aerosol property variations, with the least effect from undetected cloud fragments. This 
would also imply that particle size and/or concentration increase near detected clouds 
more gradually for undetected cloud particles than for aerosol particles. 

4.3 Effects of cloud separation distance and cloud size 
4.3.1 Effects of distance between clouds 

Let us examine the dependence of aerosol properties near clouds on the distance between 
neighboring clouds (i.e., the lengths of clear-air segments). Figure 4 shows the relative 
enhancement of attenuated backscatter y ' 532 as a function of distance to clouds for 
minimum cloud separation distances of 3 km, 1 1 km, and 35 km for all aerosol data 
(Figure 4a) and for the highest confidence data (Figure 4b). For the sake of convenience, 
our comparisons use a nonnalized backscatter, Rfx) = y 5 32 (x)/y ' 532 (20 km), instead of 
y ' 532 (jc). This comparison assumes that aerosols more than 20 km away from clouds are 
less affected by clouds (e.g., Twohy et. al., 2009). 

There are some remarkable features in Figure 4. First of all, backscatter enhancements 
are more pronounced in cloud fields with smaller distances between neighboring clouds. 
This behavior is likely the result of correlations between relative humidity and the size of 
clear-air segments. Large areas of clear sky are usually associated with local high 
atmospheric pressure and relatively dry air in which aerosols have less chance to become 
highly humidified. 
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For further examination, we define the size of a transition zone as the distance to cloud x 
where Rt(x)= 1 .05. The size of transition zones for the all data cases (Figure 4a) are 1.0 
km, 5.0 km and 7.0 km for the red, blue and black curves, respectively. These are much 
larger than their high confidence data counterparts: 0.6 km, 2.0 km and 3.0 km (Figure 
4b). In addition, the most pronounced backscatter enhancements occur within 4 km from 
clouds for the high confidence data case whereas for the ‘all aerosol’ case, noticeable 
enhancements start farther. 

The reduction of the size of transition zones between cloudy and cloud-free areas is a 
direct result of using only highest confidence aerosol data. In fact, the spatial extent of 
the transition zone is determined physically by the characteristics of clouds and their 
environment (Koren et al., 2007). Cumulus clouds, for example, may produce very 
different transition zones during different stages of their lifetime. During cloud 
formation, transition zones are usually smaller, because of the drying air produced by the 
downdraft just outside cloud boundaries. After clouds are fully developed, they start 
dissipating and produce a greatly extended transition zone that contains a mix of highly 
humidified aerosols and evaporating cloud droplets. Our CALIOP aerosol data includes 
samples from all cloud forming and dissipating stages and therefore should have a fairly 
large transition zone, as shown in Figure 4a. In contrast, the high confidence level cases 
with smaller transition zones in Figure 4b are likely associated with clouds in the forming 
stages, where the environment is relatively dry (Koren et al., 2007). 



The most pronounced backscatter enhancements observed within 5 km to clouds for the 
highest confidence data (Figure 4b) mostly reflect the aerosol swelling effects in high 
relative humidity (RH) near clouds, with the least cloud contamination effect. In study of 
the effect of RH changes on the aerosol scattering near cloud, Twohy et al. (2009) found 
a 35% - 65% scattering enhancement between 20 km and cloud edge, with more than 2/3 
of the increase occurring within 4 km to clouds. In addition, 26% - 30% of backscatter 
enhancement was observed within 5 km to clouds in the study of aerosol-cloud 
interactions using the High-Spectral-Resolution Lidar (HSRL) data over US east coast 
areas (Su et al., 2008). In contrast, backscatter enhancements in Figure 4b are about 6%, 
12.5% and 24% for different distances between clouds. The discrepancies with aircraft 
measurements are partly due to the different time and locations of measurements, as well 
as atmospheric attenuations and molecular scattering being included in Figure 4. (Note 
that the information about distance between clouds in aircraft measurements was not 
available). 

4.3.2 Effects of cloud horizontal size 

This subsection examines the relationship between near-cloud backscatter behavior and 
cloud horizontal size. It compares near-cloud properties for three cloud size ranges: 1-2 
km, 2-15 km, and greater than 15 km. Because high confidence aerosol data is less 
affected by cloud contamination and is more representative of aerosol particles, only 
highest confidence data with CAD=(-99, -100) is analyzed. Figure 5 presents y' 532 vs. 
distance to cloud relations for the three ranges of cloud horizontal size. The figure shows 
that the attenuated backscatter y' 532 behaves similarly for all three cloud-size ranges, with 
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slight differences near clouds. Noticeable differences in y ' 532 occur mostly within 1-2 km 
from cloud edges, with y ' 53 2 being slightly (~6%) smaller near small clouds. This result 
shows that horizontally larger clouds tend to have slightly stronger influences on aerosols 
in the surrounding air (perhaps because large convective clouds often have a greater 
vertical extent and can impact a thicker atmospheric layer than small convective clouds), 
but the large-scale meteorological conditions supporting large or small clouds do not 
strongly affect much on the aerosol behaviors near clouds. 

Finally, we note that clouds outside the lidar track can also influence the observed areas, 
and their influence can be mistakenly accounted for the contributions from clouds on the 
track. The errors caused by this flaw have been analyzed by a simple random cloud held 
simulation (Tackett and Girolamo, 2009) and estimated by collocated CALIOP and 
MODIS observations (Varnai and Marshak, 2011b). Both results show that including the 
off-track clouds does not affect the general near-cloud trend while only correct the 
specific values. 

5, Summary 

This paper examines near-cloud behaviors of lidar backscatters using CALIOP data of 
various confidence levels defined by Cloud- Aerosol Discrimination (CAD) scores. The 
results show that, in consistence with previous studies, lidar backscatter is enhanced near 
clouds. At the same time, the fraction of high confidence aerosol detections decreases 
near cloud edges. The results also indicate that parts of the backscatter enhancements 
come from lower confidence level data that may be cloud contaminated to some extent. 



The results also imply that using only high confidence level data reduces the impact of 
cloud contamination. 

In particular, farther away from clouds enhancement rates (gradients of attenuated 
backscatter with respect to horizontal distance) are smaller for high confidence level data 
than for all data, whereas closer to clouds enhancement rates tend to be much greater for 
high confidence level data than for all data. As a result, highest confidence level aerosol 
data yields a much narrower and sharper transition zone near clouds than the full dataset 
does. This difference between the behaviors of all data and high confidence level data 
can lead to significant differences in the interpretation of backscatter data and in retrieved 
aerosol parameters. 

The results for the highest confidence level data also show that near-cloud backscatter 
enhancements depend on the size of clear-air segments between neighboring clouds. It 
was found that the backscatter enhancement is lower for larger clear-air segments, which 
may reflect the fact that the air tends to be dryer in larger clear sky patches. In addition, 
while the near-cloud enhancements generally do not depend on cloud size, enhancements 
within 1-2 km of cloud edges are slightly stronger near larger clouds. 
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A comparison of near-cloud behaviors based on CAD Versions 2 and 3 

The CALIPSO cloud and aerosol discrimination (CAD) algorithm is probability based. A 
classification decision is made depending on what event (cloud or aerosol) is more likely 
to occur for a given set of measured parameters. Specifically, Eq. (1) is used for 
classification decisions. The PDFs in Eq. (1) are developed based on expert evaluations 
of lidar measurements for a global dataset (Liu et al., 2009). There is an overlap in the 
cloud and aerosol PDFs in parameter space where both aerosol and cloud can be possibly 
present. Outside the PDF overlap region where only cloud or aerosol can be present, a 
correct classification can be accurately made. Within the overlap region, a classification 
can still be made, but misclassifications will occur. 

Version 2 (V2) of the operational CALIOP data processing algorithm uses three- 
dimensional (3D) PDFs (as functions of backscatter, color ratio, and altitude) (Liu et al., 
2009) while Version 3 (V3) uses five-dimensional (5D) PDFs (also considering two 
additional parameters, depolarization and latitude) (Liu et al., 2010). In this 5D space, 
clouds and aerosols separate better and the PDF overlap region becomes smaller than in 
the 3D space. Therefore, the classification of clouds and aerosols has been improved, 
especially for dense dust layers that can be misclassified as cloud over/near the source 
regions in the Version 2 release (Liu et al., 2010). 

However, the V3 CAD does not behave as expected near clouds. As shown in Figure Al, 
the fraction of high confidence samples of V3 cloud-free profiles generally increases 
approaching cloud whereas the same fraction in V2 decreases remarkably as seen in 
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Figure 2b. We do not understand this V3 CAD behavior yet. We note, however, that in 
the 5D space, the overlap region between clouds and aerosols is greatly reduced. The 
developed 5D PDFs used to calculate the CAD score may not be accurate enough to 
represent well the reduced overlap region. That is, although a better classification has 
been achieved by using 5D algorithm in the V3 release, the CAD score values near 
clouds may not be accurate enough due to attenuation by overlying layers or an 
inadequate correction for the overlying layers (Liu et ah, 2010). 
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Figure 1. Effects of CAD-based data selection on the median attenuated total 532 nm 
backscatter, Y' 532 , integrated up to 3 km altitude for two ranges of distance to nearest 
cloud: from 0.3 km to 7.0 km and from 7.0 km to 20.0 km (a) and their difference (b). 
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Figure 2. Effects of CAD on the number of aerosol samples, (a) Number of samples 
under different CAD score criteria and (b) their corresponding fractions over the total 
number of aerosol samples. The break near 5 km is due to the exclusion of data from 


clear-sky segments shorter than 5 km. 
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Figure 3. CAD-dependent backscatter behavior as a function of distance to clouds, (a) 


Integrated total attenuated 532 rnn backscatter y 532 . (b) Total attenuated color ratio 
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Figure 4. Normalized backscatters R t (x ) near clouds, for three clear-sky segments > 3 


km, >11 km and >35 km, respectively, (a) all aerosol data and (b) highest confidence 


level data CAD=(-99, -100). 
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Figure 5. Near-cloud behavior of y 532 for highest confidence level data, for different 
horizontal cloud sizes of 1-2 km, 2-15 km and larger than 15 km. The horizontal axis is 
in logarithm scale. To reduce uncertainties, a longer dataset (from June 20 to Aug. 31, 
2007) is used for this figure. 
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Figure Al. Fraction of high confidence level data as a function of distance to clouds for 


V3 data. 
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